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ABSTRACT. Image 'intensities nave been processed tradT t ional ly wi thout 
much regard to how they arise. Typically they are used only to segment 
an Imaje into regions or to find edge-f ragmen ts . Image intensftfes do 
carry a ijreat deal of useful informat ion about three-dimensional aspects 
of objects and some initial attempts are made here to exploit this. An 
understanding of how images are formed and what determines the amount of 
light reflected from a point on an Object to trie viewer \s vital to such 
development. The gradient-space, popularized by Huffman and Hackworth 
is a helpful tool in this regard. 

This report describes research done at the Artificial Intelligence Labora- 
tory of the Massachusetts Institute of Technology. Support for the labora- 
tory s research is provided in part by the Advanced Research Projects 
m^T th& Department of Defense under Office of Mva I Research contract 
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AND HW FOR 50tt£Tlfi u& COMPLETELY DIFFERENT 

A case will be made for the usefulness of image Intensities or gray 
levels. Usually one would like to forget about image Intensities as soon 
as possible, extracting only edge-fragment* Or regions before going on. 
huch of the work in Image analysis has used Image intensities only to seg- 
ment Che i mage , based on differences in average Image intensity or same 
hlgher-ordef measure* A great deal of information is contained in the 
Image Intensities, however, and there are ways of exploiting this fact. 

Our approach Is based on the belief that ft rs important to understand the 
image- forming process tf one is- to construct models of the world being 
imaged. It Is not sufficient to try an assortment of statistical, compu- 
tational, or signal -processing tricks that come out of a bag of procedures 
that has proved useful in some other domain. 

Using an understanding of the Visual effects of edge imperfections and 

mutual illumination, we will be able to Suggest Enterp retat Ions of lines 
based on image Intensity profiles across edges. A "sharp peak" or 
edge-effect will imply that the edge is convex, a "roof" or triangular 
profile wfll suggest a concave edge, while a Step-transit Ion or discon- 
tinuity accompanied hy neither a sharp peak nor a roof component will most 
likely be an obscuring edge. This latter hypothesis is strengthened sig- 
nificantly if an "inverse peak" or negative edge-effect is also seen. 

Next we will show that the image intensities of regions meeting at a joint 
corresponding to an object corner allow one to determine fairly accurately 
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the orientation of each of the planes meeting at the corner. The three- 
dFrnejis Tonal structure of a polyhedral scene can thus he established with- 
out trie use of size- or support -hypotheses or a finite catalogue of models* 

Finally we will turn to curved objects and show that their shape can be 
determined from the Intensities recorded in the Imager The approach to 
this problem presented here Is supported by geometric arguments and (foes 
not depend on methods for solving First-order nonr 1 i near partial different- 
ial equations. It is instead a synthesis of the previous shape- from- shading 
method and the gradient-space approach. [4 t 2] . 

Thr- results presented here depend to a large degree on geometrFc insight 
gained by using the gradient-space approach popularized by Huffman and 
hackworth. [1,2*3)3]- Approaching the image analysis problem in the way 
proposed in this paper leads to the ability to prove or disprove that cer- 
tain features can be extracted from Images, It is not claimed, however, 
that it makes any inroads on the scene orn.il ys I S problem,. 
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DE_V110P>W'G THE TOOLS 
IHAGE FORMAT I DM: 

Our usual visual world consists of opaque bodies immersed in a transparent 
medium. iinca we cannot see into Opaqu* objects, their surfaces are im- 
portant for recognition and description purposes. This special nature of 
our visual world makes it reasonable to attempt to derive a nodal of what 
is being seen fron an Image, The dimensionalities of the two, domains match: 
On the one hand, we have tW^d3r*-nsiona I surfaces plus depth, on the other, 
two image d I men s I On s plus intensity. 

If wc are to exploit this Dbservat fon w* have to understand how image? arc 
formed. There are two parts to this problem. One deals with the two image 
dimensions and relates them to the surface coordinates, and the other deals 
With the determination of what intensity will be recorded in the image at 
each point, 

PROJECTION; 



First, l«t us look at the geometry of projection, For this purpose one 
can replace a lens with a pin-hole st Us center. Straight lines then con- 
nect points on the objects to their Images -- these 1 fnes pass through the 
pin-hole, If we let £x,y,z) be the coordinates of some point before the 
viewer, and [x'^y 1 } its image coordinates, then 

*' - (x/zH aiid y 1 - {yVz)f 
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Here f f s the separation of the fmage plane from the 1ens> It is conven i 
ent to superimpose the Fmage plane onto the object space as follows: 



U.yz) 




pinhole 



Above is the well-known perspective projection. Sometimes it is con- 
venient to consider a simpler case where objects ace very far away rela- 
tive to thetr sTie. We can Imagine loofcTng at them through a telephoto 
lensn The scene then wTll occupy e small visual angle and the df stance to 
points on the object will be almost constant In the projection equation. 



x' = (f/z )k #nd y' = (f/i Jy 



This corresponds to orthographic projection. 
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SURFACE flfilEHTAT J n W ■ 



In order to define the 1 fght flu* reflected En the fraction of the 
viewer frora a partfcular surface c |ener,t of the object, m wT T 1 have to ' 
understand the 1 l 9 ht-««r« t object-surface , ¥ r««r geometry, m particu- 
lar, the surface orientation will play a major role. 

There are varies ways of ■ PK |f vin9 the au rface orientation for a plane, 
Ve can, for example, give the equation defining the plane, or the direction 
of a vector perpend r Cu far to the surface. If an elation for the pi an * 
t5a " Uv + cz = d - CrtEn a Stable surface normal is (a.b.cj. In fact 
we can rewrite the elation {x.y,zj ■ {a , b,c) - d. To 4hc*, that any line 
In the surface Is Indeed perpendicular to the norma) so defined, consider 
any pair of points in the surface [^.zj and (x,, ,,..,). Connecting 
the, and taking dot-products *e find that ^x, , V >WV '^ b ^ " °' 

Since WE shall be Interested in curbed surf acts as w.11, ri e extend this 
method for specifying surface orientate by applying It to tang e m planes. 
That is, the orientation of the surface it a point (x Nr ,i ) i, defined 
to be the orientation of the tangent plane constructed at that point. If 
the equation of the surface U S iven as z - zfc.y), ^ can take an infini- 
tesimal step {dx, dy, di) in the Surface and find that d £ = z d* + z dv 

x y 

where i k and z y are the first partial derivatives of z *rth resp cct to * 

*nd y respectively. Clearly, the equation of the tangent plane can be 
written a. z x * * z y y - z * d (where d = z^ + ^ - z j r Ve can iraned] _ 
ately construct a local normal fz ,i p -|). 



y 
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It will be convenient to abbreviate the First partial derivatives, as p and 
q. The local normal then becomes- (p t q t -l). It is clear that orientatEon 
defined In this way has but two degrees of freedom. The quantity (p.q) 
will be called the gradient. 



local nornnat 




tangent plane 



WAGE INTENSITY; 



Hextwetum to the Image Intensity^ This will be equar to the amount of 
light reflected by Che corresponding poTnt on the abject in the direction 
of the -J Fewer > multiplied by some constant factor that depends on the param- 
eters of Che E mage- forming system. To be precise, we have to think of in- 
tensity as light flux per unit ares and correspondingly also have to con- 
sider the reflected tight per unit area as seen by the viewer. 

Now the amount of light reflected by a surface depends on its micro- struc- 
ture and the distribution of the incident light. Constructing a tangent- 
plane to the object's surface at the point undar cons 5 rk-rat i on h one Scev 
that light may be arriving fro« direct ions distributed over a hemi-spheCe. 
One can consider the contributions from each of these- directions separately 
and superimpose the results. 
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Tha important point Is that no matter how cOmplo* the distribution of 11 grit- 
sou rces* and for most kinds of surfaces, there is a unique vjluc , .0 f. Re- 
flectance ., and iiiage in can 5 icy, for a give^ nr Fentat io-n . of the Surface- 
Wc shall spend some t ln»e exploring th*t tttdJ develop the gradient-space image 
in the process - 

S INGLE POINT SOURCE: 



The simplest case is Chat of a single point-source. It is easy ta see that 

j 
the geometry of reflect Ton Fn this Case is governed by three angles, the 

incident, the emittance, and the phase anglis. The incident angle is the 

angle between the incident r*f and the local normal t the omittance angle. 

is the angle between the emitted ray and the local normal t and the phase 

angle Is the angle between the incident and emitted ray [h] . 



normal t 




Clearly the cosines of the three angles can be found simply by taking the 
dot -product of the appropriate paEr of unit vectors. The reflectivity- 
function is a measure of how much of the light incident on a Surface dc- 
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ment fs reflected in a particular direction. Roughly Speaking, it is the 
friction of the incident light reflected per unit Surface area, per unit 
solid angle, in the direction of the viewer. 

Let the illumination he E (flux/unit irca) and the resulting surface lum- 
inance in the direction of the viewer be B (flux/steradian/projected area}, 
Projected area is simply the equivalent area if the surface was not fore- 
shortened h that js ± If it was normal to the v Few-vector . The. reflectivity 
is simply defined as B/E, It If. usually wrFttan f(i,e,g). 

Note that an infinitesimal surface element, dA, captures a flux E cos(i)dA, 
since its surface normal is inclined l T relative to the Incident ray* 
Similarly, the intensity I (f I ux/steradlan) equals 8 cos(e)dA h since the pro- 
jected area is foreshortened by the Inclination of the surface normal rela- 
tive to the emitted ray. 

REFLECTIVITY FUNCTION J 



H« thciat leal models have been constructed for some surface! that allow 

an analytical determination of the reflectivity function. Such techniques 

have not proved very successful so far. 

In genera! wc may not just have a single pofnt-SOurca illuminating the 
object -- other objects around it, for example* wTll contribute to the 
incident light. In this case, One has to Integrate the product of the re- 
fleet ivity function and the incident light per unit solid angle over the 
hemi-sphere visibte from the point under consideration In order to deter- 
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mine the total light flu* reflected In the direct ton of the vi 



TYING IT ALL TOGETHER; 



ewer. 



So far we have treated geometry and intensity separately, the normal to 
the surface relates object geometry to image In tensity. The norma! ts 
defined in terms of the surface geometry, and it also appears In 
the equa-tTon for the reflected light intensity since the three angle* de- 
termining reflectivity depend on. Ft. One could flow proceed to develop 
partial differential equation* based on this observation -- it is more 
fruitful to introduce tnother tool first, gradEent-space. This will allow 
US to gain valuable intuitive Insight Into how one can exploit the detailed 
understanding of Image format ion. 

GRADIENT SPACE' 



Grad ient -space can be derived as a projection of dual -space or of the 
Gaussian sphere, but it is easier for Our purposes here to relate it direct- 
ly to surf«ce orFentatian [2]. We will concern ourselves with orthographic 
projection only, although some of tho methods can he extended to deal with 
perspect ivc. 

The mapping from surface orientation to gradient-space- "<n st ra i ght-foruard . 
If we construct a normal fp t q>-l} at a point an an object p it maps into the 
point (p P q) in gradient-space. Equivalent ly , one can imagine the normal 
placed at the origin and determine its intersection with a plane at unit dis- 
tance form the origin, if we write the equation for tfie surface z - z(x,y)^ 
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then a normal to the surface wi M he (^,2 ,-l), hIw^q z and z are the 
first partial derivative* Of z with redact to * and y respectively. 

Clearly, p = z and q « E . 

X y 

We heed to look at some examples to gain a feel for grad Iftht-space. Evi- 
dently ii plane maps Into a poTnt In gradient-Space. A second plane parallel 
to the first maps Into the same point. What plane maps TtltO the point at 
trie origin En gradient -Space? A plane with npr-pal {0,0,-1), that is, a 
plane perpendicular to the view-vector (C,Q,-|), 

Having away from the origin in gradient-Spate, one finds that the distance 
from the origin corresponds to the inclination of the plane with respect to 
the vfen-vettor — specifically, the distance from the origin equals the 
tangent of the angle between the s u r f a te- no nma 1 ei\d the vlew-vector f tan(e) r 

If irt rotate the object-space about the view-vector, we induce an equal 
rotation Of gradient-space about the origin. This allows us to line up 
points with the axes and so simplify analysis. Using this technique it is 
easy to show that the anguTar position of a point in gradient-space corres- 
ponds to the direction of steepest descent on the original surface-ate. 

Let US call the orthogonal projection of the original space, image-space, 
Usually this is all that Ts directly accessible to us. Two planes inter- 
sect in a line. Let us call the projection of this line the Image-fine. 
The two planes, of course, also correspond to two points in gradient-space. 
Thn line connecting these two points is called the gradient- line. Thus, a 
line maps into a line. The perpendicular distance of the gradient-space 
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line fncxn the origin equals the tangent of the inclination of the original 
line to th* linage plane. 

It car he shown that If the gradient-space were to be superimposed On the 
Image-Space, an image- line would be perpend icular to the corresponding 
gradient-Space line, Mackwrth's scheme for scene analysis of line-drawings 
of polyhedra depends, an this observation [1] . 

TRJ-HEORAl CODERS: 

The points In gradient-space, that correspond to the three planes meeting at 
a tri-hcdral corner, ha^e to satisfy certain constants. The lines connect- 
ing the&e points have to be perpendicular to the correspond 1 ng lines Tn 
T mage -space. 



B 



IMAGE-SPACE 




GRADIENT SPACE 



This provides us with three constaints -- not enough to fix the position of 
three points in gradient --space < Three degrees of freedom »re still undeter- 
mined, namely the position and scale of the triangle, u« shall see later 
that measuring the three Intensities provides enough information to dis- 
ambiguate the orientations of the planes, and thus allows a determination of 
the three-dimensional structure of a polyhedral scene. 
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GRADEE>1T- SPACE IMAGE; 

The amount of ] Tght reflected by a given surface element depends on its 
orientation and the distribution of I ight- sources around rt, as uell a? on 
the mature of its surface. For a given type of surface and distribution 
of U ant -sources, there is a fT*ed value of reflectance for every orienta- 
tion of the surface normal and, hence, for every point \n gradient- space. 
Image intensity is a single-valued function of p and q. tfe can think of 
thts as e gradient-space image. ThFs is not a transform of the [mage seen 
by the viewer. It Is, In fact, Independent of the Scene and a function of 
the surface properties and the light-source distribution. Note that we 
have assumed that both viewer and light-sources are far from the objects 
in the scene * 

The use of the gradient-spate diagram Is analogous to the use of the hodo- 
gram or velocity-space diagram. The later provides Insight into the motion 
of particles in force fields that is hard to obtain by algebraic reasoning 
alone, Similarly, the gradl enfc - ipace wi IT allow georaetr ic reasoning about 
surface orientation and image intensities. 



race. 



i ne 
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HM SURFACES AMD MHltT- SOURC E NEAR VlEWEK: 

Some examples will make this clear r Consider a perfect lamberttan Surf 
A perfect d Iff user ha.* the property that it looks equally brTght from all 
directions and trvat the amount of 1 fght reflected depends only on the Cos. 
of the incident angle. In order to postpone the calculation of incident, 
emittance, and phase angles from p and q For now, ue will place a single 
Hglit-SOurce near the viewer. Then the incident angle equals the emlttancfl 
.-itn;lu and is =; hi.,ly '.-,c sr.g\a b+d.^wi the surfac.* n-'ul and n r. vi & , v^lor 
Its cosine Is just the dot-product of the corresponding unit vectors. 
That is, 



COSfi) = <P^>-0-tO.O F -» _ U /y +p2 +qJ 

|(p>q>-0| I (0,0,-1)1 



'hi' ianc result foultl have 3een oJjTahitfd by rcnenberlng r>.u Lre distant 
from the origin in gradient space Is the tangent of the angle between the 
surface -norma I and the view- vector; 

/pZ + q 2 - tan(e) and cos 2 (e} ■ l/[l + tan a <e)] and e - f here. 

I" *■*.■ j lot riitlfif. I .-ii'i.-,,' jl- a functuvi .ii' i; ,it: q, we get ,"i r.-./nlrdi maximum 
Of One at the origin, and a circularly symmetric function that memoton i ca M V 
falls to irero as one goes to Infinity in gradient space This is a nice, 
smooth gradient -space image, typical of mat surfaces. 

A gTveni forage intensity corresponds to a simple locus. Tn gradient-space., a 
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clfd* centered On the origin. A measurement of image Intensity tails us 
that the surface gradient has to he one that falls on a cert^-i rjirc'lc in 
gradient -space, 

UHlFOftH ILLUMINATION: 



Note that the case of uniform illumination is quite similar to the situ- 
ation where the light-source Is near the viewer. For a start, there are 
no shadows in either case. Secondly, m both cases the reflectivity can be 
written as a function of the emittance angle alone r In fact, we can define 

an equivalent reflectivity function. 



#'{*) * / W/2 / ¥ t(.t»e*g) sin(T) dAdl 



o I 



for the uniformly illuminated surface, hie re A is the azimuth angle defined by 

cos (9) - cOs(i)cos(e) 



COS (A) ■ 



sinf i)sin(e) 



In general, the light-source Ts not likely to be near the viewer^ so 

we will have to explore the more complicated' geometry of incident and emitted 

rays for arbitrary direct Jons of incident light at the Object. 
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Contours of constant £ - COs(«)- Contour intervals are -1 unft& wide. 
This is the gradient -space Imege for objects with 1 auibe rt i an surfaces 
when there i* a iinglfc light-source near the viewer. 
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. NCI PENT, EHITTAHCE, AND PHASE AH&LE5: 

For many Surfaces the reflectance is a smooth function of the Incident, 
emi ttance, and phase angles. 



normal t 



4^ 

=( J^- source 




It Is convenient to work with the cosines of these angles, t - cob(i), 
E = co&(e), and G = cos{g) -- sine* these can be obtained! easily From dot- 
products of the three unit vectors t Suppose for now that »e have a single 
distant Hgiht-SOurco and that Et5 direct fen is given by a vector (p ± q ± -l) 

§ S 

The view- vector Is fO*0*-l), W? 



- I//, + p £ + q 2 ] E . l//l + p 2 + q Z t 



3'lC 



I = (r + p- a p + q s q)/(/l + p 2: + q 2 ./j + ^2 + q ^ 2 ] = ( 3 + p ^ p +. q q )EG 



Evidently it fs simple to calculate 1* E, and G For any point In ■gradient- 
space. In fact G Is constant given OUT assumption of orthogonal projection 
and distant i i ghfsourcc. We have already seen that the contours of constant 
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E are circles In grad I ent - Space centered on the origin- Setting I constant 
gives us a s econd -ord e r polynomial fn p and q and suggests CH»a t loci of 
constant I may be conic sections. The terminator, the line Separating 
lighted from shadowed region*,, is a straight line, obtained by setting 

i - n/i, Here i =■ i that Is, I + p p + q q = Q. Similarly, the locus 

5 S 

of I = I is the Single point p » p and q = q . 

A geometric way of constructing the loci of constant I is to think of the 
cone generated by all directions that have the same incident angle. The 
axTs, of the cone is the direction to the 1 1 ght-source fo ,q ,-lV. The 

corresponding points in gradient-space are found by Intersecting this cone 
with a plane at unit distance fron the origin. Varying values of I will 
produce tone* with varying angles. These cones will form a nested sheaf. 
The intersection of this nested sheaf with: the unit plane wilt be a nested 
set of conic section*. 

If we measure a particular image intensity, we know that the gradient of 
the corresponding surface element has to fall on a particular one of the 
conic sections. The possible normals are then confined to a cone, In this 
case this is simply a circular cOne, In the case of more general reflectivi 
ty functions, the locus of possible normals will constitute a more general 
figure called the Monge conc r 
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shadow lire 



Contours of constant L = coa(i). Contour Intervals are , 1 un its wide. 

The dr recti On to the source Ts Ep t q } - «K7,Or3K 

S S 

This- Is- the gradient-space fmage for objects with Lambertian surfaces 
when the 1 lght-50l""ce i 3 not near the viewer. 
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5FECULARIT/: 

Many Surfaces are not conplutely mat t having 30m* Specular reflect 1 err from 
the Outermost layers of their surface. ThTs Is particularly true of sur- 
faces that are snoot h on 4 microscopic scale. For specular reflection we 
have i = e and the Incident, emitted, and normal vectors are all in Lhe 
Same plane- Alternatively, we can say that I + e - g. In any case, only 
One surface orientation will he just right for reflection of the light- 
Source towards; the viewer. That is, perfect bpccular reflection contributes 
an impulse to the gradient-space Image at a particular point* 

In practice, few Surfaces have such perfect Specularity. Instead they 

I 

reflect some light En the direction slightly away from the geometrically 
correct direction [S] , (t can be shown that the COiine of the angle between 
the direction defined by perfectly specular reflection and any other direc- 
tion Is [2IE-G), This will clearly equal 1 one in the correct direction 
and fall off towards iero as one increases the arnjle to a right-angle. 
By taking various functions of (2IE-G) one can construct more or less com- 
pact specular contributions. Raising this function to some large power, 
for example, wi 1 1 do. 

A good approximation for some glossy ignite paints can be obtained by com- 
bining the usual mat component with a Specular component defined In this. 
way* For example, $(l 3 E,G) - *sS (n + 1)(2IE - G)" + (l - s)I will work. 
Here S varies between and 1 and determines the fraction of incident 
light reflected specularly he Fore penetrating the Surface, while n deter- 
mines the sharpness of the Specularity peak in the grad ten t- Spate image. 

1 
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CoiitGyr* for *6,£,(j) - ^asdi + t>(2lE - G) n + (I - S) , Th i -S it the gradient- 
space intake For a surface with both a matt and a specular component of re- 
flectivity N laminated hy a Single PO iflt-SOUrce, 
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FINDING ? AND q FROM I.E. AND G: 

In Ofdsr to explore Further the relation between the spec i f feat ion of 
Surface orientation In g rad Tent -space and the angles involved, we shall 
solve for p and q, given l t £„ and C, We have already shown that it 15 
simple to perform the opposite operation. One way uT approaching thfs, 
problem Is to try to solve the polynomial equations in p and q derived 
from the equations for 1 ± E^ and G, Th T S turns Out to be messy* but ft 
can be shown that] 

p - p' COS (8) - q 1 sin(fl) 

q » p' sin (0) + q 1 cosfe) 



Where 



A 4 • 1 + 2IEG * (I 2 + E* + G 5 ) 



P q 

cos(e) 5 — and sin(9) =- 



tV + %* K 2 + q 



2 



It 9s immediately apparent that for most VSllfftS of l„ E h and G H there are 
two solution points in gradient space. Notice that e here is the direction 
Of the light-source in grad ient-space; Lhe line connecting (p ,q ) to the. 
origin makes an angle 8 with th* p-a.*!*- So p" and q 1 are coordinates in. 
a new grad lent -space obtained after simplifying matters by rotating the 
axes until q = — the light source Is In the direction of the x'-axis. 
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The rvtat thing worth noticing about this set of equations is that if I/E 
Is constant, then p" is constant {remembering that G. Is constant anyway) k 
So the loci of constant I/E are straight lines. These lines are all para- 
llel to the terminator,, for which I - 0, This turns out to be important 
since some surfaces have constant reflectance for constant r/E. 




Contours of + (.I T E,G) - I/E, Contour intervals are .2 units wide. The 
reflectivity function For the mater (a I In the fnaria of the moon 3 s con- 
stant For I/E. 
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HETAUIC SURFACES; 

Consider next a metallic surface, a surface with a purely specular r«f !*«*,**, 

Each point in grad Tent-space corresponds to a particular direction of the surface 
■i.-j-r-a' .niil de hi nes ., direction '"r^ *ric-i incident light has to app,::.ucp 
the Object Tn order to he reflected towards the wlemr. Ir, fact, in gradient- 
space we can produce * complete map of the sphere of possible directions, as 
seen from the object. At the origin, for example, we have the direction 
towards the viewer. If we record an inters Ity in the gradient space corres- 
ponding to the intensity arriving at the object from the correspond Erg direc- 
tion We obtain a picture of the world Surrounding the object. In. fl ,ap pro- 
jection terms we have a plane projection of a Sphere wTth one pole of the 
sphere as the center of projection. Another way of looking at it Ts that 
the image we construct in this fashion is like One we would obtain by look- 
ing into a convex mirror - a metallic: paraboloid to be precise. 

What can we do with this strange image of the world surrounding the object? 
If we measure a certain intensity at a given point on the object, we can 
now Say something about the oriental Ton of the Surface at that point- Me 
cannot uniquely determine that or Ten tat Eon, but we do know that ft Is re- 
stricted to a sub-set of all possible orientations. We have One constraint 
on it -- It has to be one of the points in gradient-space where we find 
this same value of Intensity, If the world surrounding the object \ & at 
all complex, this sub -set will tefld to be very disconnected and complex 
and not much help in recovering the shape- directly. There are exceptions — 
light-sources, for example, tend to be compact and very bright, correspond- 
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ing to d*f ifi i tc casy-to-locate points Tfl gra<J i en t- space . For pclms with 
such high reflected Hight Intensity 111 the image we can often locally de- 
\'.'rw \ -■■;.■ \'v- surface nomal uniquely. 

We nave new developed nethods for constructing gradient-space linages for 
various Surfaces end c i b? r i hut tons of f ight-sources- The latter is done 
3 r«ipl y by s.upBrtmpa&ing the resuTts in gradient-space for each light- 
source In turn. We will now turn to a minor flaw in this approach and 
attempt fl partial analysis of mutual i 1 luminatton. 





>? 



si 
I 
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Gradient -Space ina^C for a Metallic Object in the center of a large wire 
cube- Equiwalefltly One can t h f rik Of it as Lhc reflection of" the wire cube 
in a paraboloid with a specularly-ref lect ■ ng Surface. 
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HU.THAL ILLUMINATION: 



The gradient-space image is based on the assumption that the Viewer and 
all light sources are distant from the object. Only under these assump- 
tions can W4 associate a unique value of image intensity wEth every surface 
or rotation. ,f the SCcnE con5ists of , single conVeR D bj> ct tnese assump- 
tions may be satisfied, but when there are several highly reflective ob- 
jects plated near one another, mutuaf illumination may become important. 
That is, the distribution of incident light no longer defends only on direc- 
tion only, but is a function of position as well. The general case is very 
difficult to deal with and we shall study only some Ideal lud situations 
applicable to scenes made up of polyhedra. There are two- primary effects 
of mutual illumination: a reduction in contrast between faces, and the 
appearance of shading or gradation of light On images of plane surfaces, 
tn the absence of this effect, we would eKpecE plane Surfaces to have 
polygonal images of uniform intensity since all poTnts on them have the 
same orientation, 

I¥B SEMI- INFINITE FLAMES: 

First let US Consider a highly Idealized situation where we have two Semi- 
Infinite planes joined at right angles, and a distant light-source. Let 

thft incident fays make an angle a with respect to one of the planes. Further 
assume that the surfaces reflect a fraction r of the light falling on them, 
and i hat the illumination provided by the source Is E (light flux/unit area). 
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Picking any point on One of the half-planes, we find that one-half of its 
hemisphere Of directions is occupied by the either plane, SO one-half of 
the light radiated from this point will hit the other plane, while one-half 
will be lost. Since both planes ar« semi - inf i ni te x the geometry of thj» 
does not depend on how far from the corner we are. Wow, the light incident 
at any point Is made up of two componen t s , that received directly fron the 
source and ih-JL reflected fr-fvij the other plane. It Fs not hard to see that 
the intensity on One plane will not vary with d i stance from the corner -- 
a point receives reflected light from one-half of Its hemisphere of direc- 
tions- ng matter how far from the corner it is. Put another way, there is 
no natural scale factor for a fluctuation in intensity- Let the illumina- 
tion of the planes be E, and E ? flight flux/unit area). 

E, - *jE 5 + E cosU) 



l 2 ' ^ E l + E sin ^ 
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So-I^ing far E ] and E,, One gets; 

E| = E[tOs{a) + Hr sln( Q )]/[| - fer) 2 ] 

e 2 = Et»rn{«> + igr C0£ f a )j/[j _ ^ r) 2] 

Had we ignored the effects of mutual II I taxation we would have found 
E, = E cosfa) ahd E^ = E S in{«), Clearly the effect Increases with In- 
creases in reflectance r } ft \i not significant for dark surF ace5 . When 
the planer are equally illuminated, for a - v/k t we have: 

E, • £ 2 - (E/^)/(l - V) 

Uhen r = |, this !e twice the I 1 1 mn rut ron and hence twine the brightness 
that we would have obtained in the absence of mutual nomination. 

If the angle between the two planes Is varied, ane f Ends that the effect 
gets larger and larger a* the angle gets iw e and mrc acutei Qn ^ can 
gel arbitrary "ampl if ication" by choosing the angle small enough. Con- 
versely, for angles larger than */Z. the effect is less pronounced. 

In the above derivation we have not made very specific assumptions about 
the angular distribution of reflected light, Just that it does not depend 
on where the incident ray conies from and that it is symmetrical about the 
normal. So a lambertian surf** would be Included, while a highly specular 
one would not. Indeed, the effect is | e5S pronounced for surfaces with a 
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high specular component of reflect ion, since most of the light Is. bounced 
back at the source upon second reflection. 

Another important thing to note Is. that rf the planes are not Infinite, 
the above calculations apply approximately at least close to the corner. 
For finite planes we expect a variation of intensity as a function of 

distance from the comer, but asymptotically* as One approaches the corner, 
the results derived here will apply. 

TVO TRUNCATED PLANES: 



If the planes are of finite extent, the geometry becomes quite complex, but* 
If one allow* them to be TnfTnrte along their Tin* of intersection and 
truncates them Only in the direction perpendicular to this* one can develop 
on Integral equation. Suppose they both extend a distance L from the corner, 
and &re joined at right-angle* and that a = tiJU, This produces a particular- 
ly simple, for™ of this Integral equation — which nevertheless I have been 
unahle to solve analytically- Numerical methods show that the resultant 
illumination falls off monotonical ly from the corner T that the value at 
the corner is indeed what we predicted in the previous sect ion y and that 
near the corner, the fa II -off Ts governed by a term In -(x/L)* 1 ' ** . 
For r =• 1 , For eKa-nple, this contains, the square-root of (x/L) and there 
Is thus a cusp In the Function at the corner- (Here x T $ the distance 
along the plane from the edge where the planes meet). 
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Surface luminance platted versus fractional distance from a right-angle 
corner. The curves are for reflectances of .1, A, .6 t .8, arid 1.0. 
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Surface luminance plotted versus (x/l) to illustrate asymptotic 

behavior near the corner. The curves correspond to reflectances of 
.2, .*, .6, .&, and 1,0, 
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THE MAIN RESULTS 



THE SEMANTICS OF EDCE-PBQFUE5: 

If polyhedral object* were perfect, there was no nutual i Mum [nation N 
image sensori were perfect and l^ht sources dfstant from the scene, 
images of polyhedral objects would be diyldcd Into polygonal areas f with 
intensity uniform Inside each polygon. 1 1 i «, we I I known that thrre is 
variation of [mage intensity within these polygonal area* in real images 
and that an intensity profile taken across an *doe separating two such 
polygonal regions does not simply have a step- shaped transition \r> inten- 
sity. Herskovitz and B-inford determined experimentally that the most 
conwon edge transitions are step-, peak-, and roof-shaped E7I . This ha* 
SO far teen considered no more than a nyisancc, since It complroates the 
process of finding edges. Were Wc will dfscuss the interpretation of these 
profiles in terms of the three-d i mens ions 1 aspects of the scene. 




STEP 




PEAK 



ROOF 
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IMPERFECTIONS OF POLYHEDRAL EDGES: 

A perfect polyhedron has a discontinuity In surface normal at an edge. 
In practice edges are rounded off somewhat- A cros S - set t 1 On through the 
object's edge Shows that the surface normal varies ShlOGtbly From Dne value 
to Che Other and takes On values that are 1 linear combinations of the sur- 
face normals of the two adjoining planes. 




What does this noe.in in terns of reflected light intensfty? Instead of a 
sudden jump of intensity from a value correspond Ing to. the one Surface noriral 
to the other, the intensity varies smoothly, The important point Is that it 
may take on values outside the range of values defined by the two planes. 
The best way to see this Fs to consider the situation in gradient-space. 
TI:k iv.':> planes define two points in gradient-s,pace and tangent planes on 
the corner correspond to points on the line connecting these two points. 
If the image intensity is higher for a point somewhere on this line* we w 1 9 1 
see a peak in the Intensity profile across the edge. 

So T if we find an ed^e-profile with a peak-shape or a step with a peak super- 
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Iriipused, it is most likely that the line should be labelled convex. The 
converse is not true h 311 edge may be convex and not give rise to a peak, 
if the line connecting the two poEntS En gradient space has Intensity 
■-'.■ir>r n rj monotonical ly along Its, length* Th* identification is also not 
completely certain since under peculiar lighting Conditions and with ob- 
jects that have acute angles between adjacent faces, a peak may appear at 
an obscuring edge. 

Hot Ice that the peak is quite compact* since Tt only extends as far as 
the rounded -off edge does. 

At a corfier, where the planes aieet* we find that surface imperfec- 
tions provide surface normals that are linear comb i na t r on 5 of the three nor- 
mals corresponding to the three planes. In gradfent space this corresponds 
to points tn the triangle connecting the three poFntS correspond incj to the 
planes- If this triangle contains a maximum [n [mage intensity we expect 
to -see a hEgh-light right on the corner. 



-3*- 




lmag& of tri-hedral corner afid correspond! I>g gradient- S.pacc diagram. 
The image intensity p fQ f i 1 e across the. edge between face A and face 
B will have a peak or highlight. The others will not. 
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MUTUAL iLLUMINATIONr 

We have already seejl that mutual 11 1 u-ti tnat Ton gives rise to intensity vari- 
ations on pUiar surfaces, The intensity f a J i s off a* ens moves awa y f reffl 
the corner. Near the comer, this fa I I -off is appro* imate I > 
linear. Notice that this affect, the Jnt B nsity profile over a large dis- 
tance fron the edge, quite unlike the sharp peak found du* to edg e imper- 
fections. Clearfy, if „« fi^i a roof-shaped profile or step wTth a roof- 
shape super Imposed we should conifer labelling the edge concave. 

The identification Ts not perfectly certain, though, slrTCC some imping 
devTce defects can produce- a similar effect. Image dissectors, for example, 
Suffer from a great deal of scattering and this has the effect that areas 
further from a dark background are brighter. So one l»y see a Smoothed 
versi&n of a roof-shape in the middle of a bright scene against a dark 
background. Expel* intent* t ion with high-quality imaqe input devices such 
a?. ihL- FIK-:JiodB inTrror-dafTectron system has confirmed thftt this li en 
artifact Introduced by the image dissector. 

Further, when the light -source \s close to the scene, stgn [ f leant gradients 
can appear on planar surfaces as pointed out by Herskovit* fr &Snford [7]. 
Lastly, the roof-shaped profiles on the two surface may be due to mutual 
Illumination with other surfaces, not each other, Meyer tbel ess, a roof- 
Sh*p c d profile does usually suggest a concave edge. 



■3£- 



O&SCUftATIOH: 

Step- shaped Intensity profiles most often occur where objects ob^Cyi-e 
one another* although they can be Found with convex and somet imo^ con- 
cave edges as well. If the obscuring surface adjoins a self-shadowed 
surface, however, edge [mper feet Ions will produce a negative peak on the 
profile, &Tnce the Hie eonnectTng the points corresponding to the two 
surface* fn gradient-space then passes through the terminator. So a nega- 
tive peak or a step with a superimposed negative peak strongly suggests 
Obscurat ion* It is unfortunately impossible to tell which side is the 
Ofc &C u if r ng plane. 
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Generatton of a negative peak at an obsturtng edoc facing a wav 
from the M-ght-sourftE. 
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PETERfHHlHS THE THREE- PIHEHSIOHAL STRUCTURE QF F&LYHEOftAL StEMtS: 

The approach invented by Haekworth for understanding line-drawings of 
polyhedra allows one to take Into account some of title quantitative aspects 
of the three-dimensional geometry of scenes [2], It does not,, however, 
allow One to determine fully the orientation of all the planes.. The scale 
and position of the gradient-space diagram is undetermined by his technique, 
To illustrate, consider a single trihEdraJ corner. Here we know that the 
three points in gradient-space that represent the three planes meeting at 
the corner have to satisfy certain constraints. Specif leal ly t they must 
He on three lines perpendicular to the image- 1 ines. 





It takes sIk parameters to specify the position of three points on a plane, 
so we still have three degrees of freedom after introducing these constraints- 
Measuring the three image intensive* of the planes Supplies another three. 
The constraints are due to the fact that the points in gradient-Spate have 
to He On the right contours of image intensity. The triangle can La 
stretched and moved until the points correspond to the correct image inten- 
sities as measured for the three planes. Since thts process corresponds 
to solving three non-1 inear equations for three unknowns, we can expect a 
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finite number Of solutions. Often there- ate but one or two;, some 

can be eliminated From prior knowledge of what Is to be Expected in th* 

scene. 

When , IHirc than three planes meet at a corner, the situation is even more 
constrained ■*- the equations are over-detennl ned. Conversely, One cannot 
do much with just two plane* meeting at a.n edge, since there are too few 
equations, and an Infinite number of solutions exist t as one might eapect. 

The possible ambiguity at a trf-hedraT corner is not very serious when one 
considers that in a typical scene there will be many "connect" edges, eTther 
convex or concave as determined by Macfcworth's program. Usually the aver- 
all constraint! will a II ow on I y on* interpretation that Is consistent, 
A practical difficulty is that It la Unclear what search Strategy will lead 
One efficiently to this interpretation. 

Measurements of image intensity are not very precise and surfaces have 
properties that vary From point to point and with handling. Vie Cannot 
expect this method to be extremely accurate in pinning down surface orienta- 
tion. The fact that for a typical scene the equations will b* ov*r-deter- 

mihed allows a 1 east- squares approach which may help to Improve matters a 
little. 

The idea of stretching! and shifting can be generalized to smooth surfaces. 
We know that the image of a paraholoTd Is the gradient-space Image, ff 
we can stretch and shift a reel Image of some object to fit thil pattern 
Of intensity d I itr I but ion we can determine its surface shape by applying 
the Inverse stretching and shifting to the paraboloid. 
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LJM AR TQP&ERAPHV: 

When vEtwed from a great distance, the fliatc-rial in Che marEa of the moon 
has a pertfcularry interesting rsf Im; t iy ! ty Function. First, rote that 
the lunar phase is the angle at the moon between the 1 ighr~souree (sun) 
and the viewer (earth). This 15 obviously the angle we call g, and ex- 
plains -why we use the term phase art^le for g. For constant phase- angle, 
detailed measurements using surface element^ WnOSc projected area as seen 
from the- source is a constant multiple of the projected area as seen hy the 
viewer, have shown that all such surface elements have the same reflectance. 
But the area appears foreshortened! by CCS ( i ) and cos (e) as seen by the source 

and the viewer respect ivelv- Hence the reflectivity function Is constant for 
constant cos ( 1 )/cos (e) - I /E (for fr*ed G). 

Each surFace element scatters lit;ht uniformly into Us hemisphere of 
dTrectEonS, quite unlike the l«mb*ft<an surFace, which favors directions 
normal to its surface. This is not an isolated incident. The ■surfaces. 
of other rocky, dusty objects when vfewed from great distances appear to 
have sJmTler properUes. The Surface of the planet Kerc-ury, for example, 
and perhaps Mars* as well as some asterpEds and atmosphere -free satellites 
fit this pattern, Surfaces with reflectance a function of I/E thus form 
third species we should add to ma,t surfaces where the reflectance is a 
function Of I and glossy surfaces where the reflectance Ts a function of 
(21E-&J* 



_L UMAft ftEFiECTIVITV FJNCTrOjh 

Returning to the lunar surface* nje find an early forniuia due to Lorrire I 
Seel fnger [4 J : 



r (i/eJ 



(I/E) + >.(£) 



Here ^o fS a constant and the function a(G) is defined by an empirically 
determined table, A somewhat not* satisfactory f i s to the data is provided 
by a formula of Fesenkov's [fej : 



r ti/E)[p + cos 2 (a/z)3 



(l/E> + * M + tan 2 (A/2)] 



Uhere r fl and X fl are constants and tanfo) • - [i/E-Cj/ZTE 2 , By the way, 
tan fa) - -p 1 , This formula is also supported by a t bee re t i ca 1 model of 
the surface due to Hapke, Koto that given I. E, and G, it If straight- 
forward to calculated the expected reflectance. We need to 90 in the 
reverie direction and SO We for I/E given G and the reflectance as measured 
by the Image intensity. While it nay be bard to Invert the above equation 
analytically, it should he clear that hy SOl r* iterative, Interpolation, 
or hi I l-cl imbi-ng scheme, one can salve for I/E. We shall ignore for now 
the ambiguities that arise [f there is raOre than one solution. 
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LUHAR GRADIEWT-SPACE IMAGE: 

Next, we ask what the gradient-space inage looks Mke for the lunar surface 
illuminated by a afngle point -source. The -contours of constant intensity 
\n gradient-space will be lines of constant l/E. But the contours of 
constant l/E are Straight 1 inesl So the gradient -Space imdcjc can be generated 
from • single curve by shifting it along a stra ight-1 in* -- the shadort-line, 
for example- The contour lines are perpendicular to the -direction defined 
by the position of the source (that is, the Hue from the origin to p ,q ). 

Ei 5 

Now what information does ti single measurement of image Intensity provfde? 
It tells us that the gradient has to be on a particular straight line. 
Again, we shall ignore for the iroment the passible existence of more than one 
contour for a given (intensity. What we would like to know of course Is the 
orientation of the surface element. We cannot determine completely that locally, 
but we can tell what its component will be in one direction, the direction 
perpendicular to the contour lines. We. can tell nothing about it in the 
direction at right-angles to this favored direction. In Fact* knowing l/E and 
G determines p", as previously defined and tells up nothing about ^ , . 
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This fevered direct ion lies in the plane defined b-y the sour**, the 

V Tewer, and the surface element under cons Herat ion. If erne wishes, on* 

can simplify otters by rotating the viewer's coordinate gy Slem * yatem mtM 

the x axl* | leJ in thFi plane as well. Then ^ . 0j art the «ntours of 

dOft*tant Intensity in gradrent-space are ail vertical lines, Evidently. 

an image intensity measurement determines the slope of the surface in the 

x< direction, without teH mg us anyone about the slgp c In the v 1 direction, 
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We are now ready to integrate out the surface by advancing in the direction 
in which we can locally determine the surface slope. 
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FIWCUNG A SURFACE PROFILE BY INTEGRATION 
We have; 



pi = 4z = i/E - G 

ds A -~~ d 3 



The distance = from soite starting point is meatured In the object coor- 
dinate system and is related to the distance along fl-e project ton of this 



curve Jn the image by s 1 « $ f f / 2 }, 



dz_ 


_ f I/E - G 


ds 


* /r "^ 



Integrating, we get: 



"■■»-*.*rJ ^- 



-- ■ 



°-° A^W 



Where I/E is found from G and the image Intensity bfjt',y'> by * : 

G 



I/E - f G [b(x\y'|] 

Starting anywhere in the i.nage, we Can integrate along a particular Tine 
and find Che relative elevation of the corresponding points On the object. 

The curves traced «Jt Or, the object En this fashion ftp. Called characterises . 

tneir projection in the image piane are tailed b4se characteristic. | t is 
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cl ear that the base Characterise id her* »tm parallel Straight Unes in 

th* image. Independent of the Object's shape. 

FJWPIHS THE WHOLE SURFfltEi_ 

Ue can explore the whole Emage by choosing sufficient starting points along a 
line at an angle to the favored direction. In this way we obtain the surface 

Shape over the whole area recorded in the image, 




Thert is nothing to relate the integrals Obtained along adjacent characteristics 
Fn the image, since we cannot determine th* gradient in this direction. We ha« 
to know an Initial curve, C-r use assumptions of reascmab 1 a smoothness. 
Alternatively, we can perform ii second surface calculation, from an [111390 
taken with a different source-surf ace -observer gconetry. Ip this case, 
we will obtain solutions along 1 Tnes cross tng the Surface at a different 
angle and can SO tie the two solutions together. This \i not quite as 
useful as one might think at first, since it does not apply to pFctures 
taken From earth. The plane of the Sun,, moon h ana earth varies little f\ 



roni 
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the ecliptic plane. The lines of integration in the image will vary little 
in Inclination. This ldlea does work for pictures taken close to the mocul- 
to the moon. 

AMBIGUITY IN LOCAL GRADIENT: 



What if more than r?ne cnn fou r in gradient -space corresponds to a given in- 
tensity? Then we cannot tell Ideally which gradient to apply. If we are 
integrating along some curve, however, Chi S I? not a problem, since we may 
assume that there is little change in yradient over small distances and 
pick the one close to the gradient last used. This assumption f smooth- 
• i u ■-■ ■.■ .■ ,■ ■-' .■ ■» i l- w " t h u n c - ena " n i n g p ■■ ob Is "1 : •;> i, . ' ' i. ■ ; i : i ■■.'. ■ t ■.■ i f w; ■■.-, ;i p roa c h lv 
fiwxirmjnl of intensity in gradient-space and then enter are** of lower Inten- 
sity* Which side of the local maximum do -we slide down? This Is an 
ambiguity which cannot be resolved locally, and the solution has to be 
terminated at thTa point. Under certain lighting condi t ions the I mage will 
be divide c in re. iv.o. i nr- ■, ir\ \<iv each of which we can find a s a 1 l- t i cm- . T-k- 
reglons will be separated by amfcigui ty edges , which cannot be crossed 
without making an arbitrary choice* 



,b<xyj 
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LQU SJH-AHGLES : 



This problem can be entirely avoided If one deals only with pictures taken 
at low sun-ingles, since the gradient is, then a single valued function of 
image intensity. This is a good idea in any case, since the accuracy of 
the reconstruction will depend On how accurately One can determine the 
gradient, which in turn depends on the spacing of the contour lines In 
gradient-Space. If they arc ctose together, this accuracy will be hlfh^ 
near a maximum* on the Other hand, it will be low. It is easy to convince 
oneself that pictures taken at law sun-angle have "better contrast," show 
the "relief in more detail 11 * and are "easier to interpret". 
There is another reason-. <"or interest in images obtained under condition* 
of low iun-angle. Near the shadnw-1 ine in gradient-Space > the contours 
of constant- intensity ar* nearly Straight lines even i f we are not dealing 
with the Special reflectivity function for the lunar material! An early 
solution to the problem of determining the shape of lunar hills made use 
of this fact by Integrating along lines perpend icular to the terminator [5] 

..DEALING WITH 5MftDuUS: 

Working at low sun-angle£ introduces another problem of course, since 
shadows arc likely to appear. For t una t* I y , they are easy to deal with 
since wc can simply trace the line in the image until we again see a 
lighted area. Since we know the direction of the rays from the source 
we can easily determine the position of the first lighted point. The 
integration is then continued From there. In fact, no special attention 
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has to be paid to this problem, sTnce 4 surface element oriented for gra?rng 
incidence of light will already have the correct slope. Thus simply looking 
up the a fope for zero intensity and integrating with Eh fa value will do. 




Some portion of the surface yf course will not be explored because of 
snadows. hast of this area wi 11 be covered If one tafces, one picture just 
after "sun-rise" and one just hefore "sun-set". 



GE NEPALI t AT I OH TQ PERSPECTIVE PROJECT I QN ; 

All along we have assumed orthographic projection — looking at the surface 
from a great distance with a telephoto lens. In practice, this is an un- 
reasonable assumption for pictures taken by artificial satellites near 
the surface. The first thirty that changes fn the more general cage of per- 
spective projection is that the Sun-su r face-viewer plane Es no longer the 
same for all portions of the surface imaged. Since it is this plane which 
determines the 1 FneS along which we integrate, we can expect thjt the 
lines of integration w i 1 1 no longer be parallel* Instead they all converge 
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on the anti-solar poTnt ~ that fi, the point in the image which corresponds 
to a direct 3 on directly Opposite to the direction towards the source. 




The next change is that z is no longer constant in the projection equation, 
Sos 1 = Hs/z). Hence, 

p ' . *= - L lL_ . l/t - 6 
ds z ds p ^1 - ti 2 



We tan no longer simply integrate- But ft Is easy to sol^e the above differ- 
ential equation for z by separating terms: 



log(z) = I / T/E " G ds' and ^ z(s'] = : e 
F 1 - C2 * 



i^-A- ds . 



/r- ? 



Finally, note that the phase angle g 1 5 no longer constant. This has to 
be taken into account when calculating l/E from the measured Image Intensity, 
Cn the whole, the process is still very sElttple. The paths of Integration 
are pre -determined straight lines In the image -- radiating from the antT- 
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solar point. At each poEnt we measure the image intensity determine what 
value Of l/Ewlll give Plst to this [mage EntensEty, Then we calculate 
the corresponding slope along the straight I Trie and take a small step. 
Repeating for all lints crossing the Image we obtain the surface elevation 
at all points in the Emage, 

The sane result could have been Obtained by a very painful algebraic 
method [6]. 



A HOTE ON ACCURACY ; 

Since inane intensities can only be determined with rather limited precision,, 
on^ must efcpcct the calculation of surface coordinates to suffer from errors 
that may accumulate along characteristics. A "sharpening" method that relates 
adjacent characteristics can reduce these errors somewhat [h] , It further 
appears that an objects shape Is better described by the orientations of 
Its surface normals than by distances f^om the viewer to points on Its surface, 
in part this may be because distances to the Surface undergo a more complicated 
transformation when the object is rotated than do Surface normal directions. 
Note that the calculation of surface normals Is not Subject to the cumulative 
errors mentioned. 

Frnally h it should be pointed out that the precise determination of the 
surface Shape Is not the main Impetus for the devlopment presented here. 
The Understanding of how image intensities are determined by the object t 
the lighting and the image forming system is of more Importance and may 
lead to interesting heuristic methods. 
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GENERAL ftEFLEET I tf IT¥ FUNCTIONS: 



on- 

<.in 



The single wet hod developed for lunar topography does not apply If the 
contour* Of constant Intensity in gradient-space are not parallel straight 
Hr.es. We shall still be able to trace along the surface, but the direction 
Wft take at each point will now depend on the image and will change along- 
the profile. The base Characteristics wi I I no longer be pre-determined 
straight lines in the Image. At each point on a characteristic Curve we 
Shall find that the solution Can be continued only In a particular direct! 
it will also appear that we wF tl need more information to start a solutE 
and shall have CO carry along more Information as we proved. Reasoning 
from the gradient -space diagram can be augmented here by some algebraic 
manipulation. 

Let a(p,q) be the Fntensfty corresponding to a surface element with a 
gradient (p,o). Let bU.y) be the intensity recorded in the image at 
the point ( Xi y). Then, for a particular surface element. We must haver 

a(p,q) = h(x P v) 

ftow suppose we want to proceed In a manner analogous to the method 
developed earlier by taking * siraM slcp Cd ^ dv) in t|w imag ^ |t ^ ^ 
that we can calculate the corresponding change in 2 as follows: 



dz = 2 dx + ? dy = p dx + q dy 
To do this we need the values Of p and a. A* we integrate Out the curve 



ear 
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we a Ho have to keep track of the values of the- $r ad lent* We can calculate 
the increments in p and q by: 

dp " P v dH + P„ dv anJ d q - q dx + q dy 

At first, we appear to be getting into more difficulty, sFnet now 

we need to know p ♦ p - q end q , In order to determine these unknowns 
r x y Jt y 

we will differentiate the basic equatTon a(p h q) = b(x.y) with, respect to 
x and y - 

a s + a q =■ b and a p + a__ Q m b 

p r x q ^ * p r y q y y 

While these equations contain the right unknown*, there are only two 

equations, not enough to solve For three unknowns. Note, however, that 

we do not really need the individual values I We are only after the linear 

combinations Cp dx + p dy) and (q dx + q dy) . 
X y x y 

We have to choose the direction of the small Step (dx^dy) property to 
allow the determination of these quantities. There Fe only one SUth direc- 
tion. Let (dx.dy) - (a„ h a )ds + then (dp.dq) - (b ,b )ds. This is the 

p q k t 

solution we were after, Summarizing, we have five ordinary dTFferertt lal 
equal Tons: 

k = a , y - a , £ ■ pa + qa , p = b , and q - b 

p q p p * r 

Here the dot denotes d i f fercnt (at ion with respect to s ( a parameter that 
varies along the solution curve r 



5J- 



INTERPRETATION IN TEP>*5 OF THE GftAfr I EHT-SPAC£ ; 



v* 



As we solve along a articular characteristic curve on the object, 
simultaneously tret* out a base characterised Ia the image and a curve 
in gradient-space. At each point In the solution w* wilt know which point 
in the Image and trflich point in the gradient-Spece the surface element 
under consideration corresponds to. The intensity in the real iiuage and 
in the gradient-space Image must, of course „ he the iame. The paths in the 
two spaces arc related In a peculiar manner. The step we take in the i*nage 
will be perpendicular to the contour 1 n gradient -space and the step we 
take lit gradient-space wi I f he perpendicular to th* intensity contour in 
the real image. 





GENERAL I ZATIPW TO HEAR SOURCE AHP HEAR VlEdEft: 

The ls*t solution method,. yhTle correct for arbitrary reflectivity functions, 
Still assumes orthographic project Ton and a distant source. This is a good 
approximation for many practical cases. In order to take Into account the 
ftf facts of the nearness of the. source and the viewer, we have to df&card 
the gradient-space diagram, since It is based, on the. assumption of constant 
phase angle- The problem tan Still be tackled by algebraic manipulation, 
much as the last solution, lit turns out that one is really trying to solve 
a first order non-linear partial differential equation in tws independent 
variables. The well-known solution involves convert 1 rvg this equation into 
five ordinary differential equations, quite like the ones we obtained in 
the last Section [M] - 



I wish to thank Kathy E. Van Sant for art early version of the numerical 
solution for the truncated-plane mutual-l 1 lumlnatlon problem and Karen 
Prendergast for preparing the drawings. 
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MATHEMflTIC AL DETAILS 
DUAL- 5 PACE: 



One approach to gradient-space, is to consider 1 t as a projection of dual 
space [1,2,5], Dual -spac* is a thrce-dipmnsional entity obtained by mapping 
planes into point? and points into planes. A point of course can be speci- 
fied as a vector {K,y,z). A plane also can he defined in term? of a vector 
(a,h,c)- The plane consists of points which satisfy the equation; 

>{x,y,z) - (a,b,c) - I or ax + by + cz - 1 

ft is clear that a plane in one space can be napped into a point In the 
other and that, conversely* a point can be napped into a plane. These 

operations are reveraJ hi e T that ls T If we start with a plane, find the 
correspond in$ point in dual-spa.ee, we can ftHp this point back into the original 

plane- 

What about lines? Lines can be thought of either as the intersection of two 
planes or as connections between two points. Thus, the dual of a line, con- 
sidered to be formed by the intersection of two planes, can be construed to 
be the line connecting the two points in dual-space that correspond to these 
two planes, A line also can be associated with the family of all planes 
passing through it -- its dual w[H be the line formed by mapping all of 
these planes into points. 

What docs the corner of a polyhedron correspond to in dual-space? First 

o* al 1 r a corner is a point, so It must map into a plane. Secondly, It lies 

in each of the planes intersecting to form Lhc corner, so its dual must eon- 
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taTn all oF the points corresponding to these planes. The dual of s corner 
H the plane defined by the point* correspond rng to the planes that inter- 
sex t to forn the corner. The edges of the object meeting at the corner nap. 

into 1 \pc-z connecting these points. 

The Object-space is not directly accessible to us. Since we have Only a 
projection of it, the image-space. We cannot einpect to arrive at the re- 
sults In dual space simply and directly -- but it turns out that a very 
useful projection of dual -space exists. 

Given a point (a,b,c) In dual-space, one can define TtS projectton Into 
gradient-Space as (-a/e.-b/c). This -^ a perspective projection. How is 
this reEated to the original object-space? Let a plane In object-space be 
defined as a* + by + ci - 1 , This can also be written: 

z - (-a/c)x -i- Hb/c)y + (t/c) 

It is clear now why (p.q) « (-a/c, -b/c) la called the gradient of the plane. 
\n fact, p - ^ and q = z the first partial derivatives of z wjth respect 

to X and y respectively. 

THE GAUSS I AH SPHERE; 

Another convenient way to talk about directions is by way of a unit sphere 
surrounding the point in question [3,9]. Points On the sphere then define 
Spec tf To directions, This representation is very convenient for some purposes 
since some useful Invariants exist on the surface of this sphere which are 
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(ost En project ton r Hufflrtan uses this to advantage En analyzing developable 
surfaces [3l^ For our purposes N however, a planar repre&cntat ion is more con- 
venfent, Grad fen t -spate Ts sTmply a projection of the Gaui:>ian :ip-iere> 
wtth the center of the sphere Acting as the center of projection and the 
projection being constructed onto a plane tangent to the sphere. 



I 



ORIGIN Of 
GRADIENT SPACE 




GAUSSIAN 
SPHERE 
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THE SRA&IEWT-LIHE IS PEHPEHDICULAft TO THE IHftGE-UHE: 
Consider two planes defined by the equations: 

a [* + bjV + c^ = d ( and a z x + b y + c z = d 

These planas hiave normals fs^bpt,) and (a 2 ,b 2P c 2 ) respectively. The 
planes intersect in a line. The direction of this 1 Tne tan be found by 
taking the cross-products of tha two normals. This follows from the fact 
that the line of intersection certainly has to be in both planes and, hence . 
perpendicular to both normals. The cross -product turns out to be 

(b i c z _l> 2 e rVr a i c z* a iWi J ' 

The image- line is the orthogonal projection of the line of Intersect Ton, 
rt* direction is limply (h^-b^e ip^c,"*,^) . 

The two planes map Tnto the pofnts h^e,,^,^,) and (-a^.-b^) in 
gradient-space. The line connecting these two points \i trie gradient- lint. 
It* direction can be found by subtraction to be fa „/c,-a,/c, ,b*/c 4 -b T /cJ , 

*- i! I 2 2 

In order to establish that the gradient- 1 In* so defined is perpendicular 
to tha image-line, we have to show that the dot-products of their respective 
directions is zero. 

(b l C 2'Vl' a 2 C r a | C 2 J '* a 2 t r a | C 2* b 2 c r b l c 2 )/(c l t 2 ) = Q 

The tuo lines are thus perpendicular. The saute result can be developed using 
only geometric reasoning. 
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I NTEG1WL OF cos (a). OVER A HEHtSPHERE: 

To provide for the correct Scaling of tne specular component of reflected 
lljht we reed the integral of co& n (e) over the Hrtfflisphsre £ g * tt/2. 




The area of ttw strip on the surface of the hemisphere is 2*R ? $ 1 n (fi)d-". 
Integrating, we get: 



_f o JTr« 2 5in(e)cos n (e}de 

2ttR 5 J o cos n {*) S 3n(8>dS 
n + 1 J° 



:■ 



This fs l/(n + l) of the surface area of the hemisphere. 
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THF OFF-$?ttUI_ARj_TY ANGLE: 

For surfaces with a specular component of reflect I vlCy one needs to know 
the angle between the reflected ray and the 1 fne of sight [8]. This angle 
can he found by simple application of some result* of spherical yeunctry. 



SPECULAR 

REFLECTION 



VIEWER 



NORMAL 




SOURCE 

Here A is called the azimuth angle- 

We are given K b, and g„ and have to find the angle 1. 

costs) » cos{i) cos(e) + sin CD sin fa) cosf* - A) 

cos(g) = cos{3) cos (el ■+ s5n(]) gln(e) cos [A) 



Clearly, cosl];;) - 2coe(p) cos (*) - cos{g) - ZIE - C 

If 2IE - G - k and c = c/tk + G) h then, (p - p c) 2 * ( q - q, e ) 2 * T * k * 

So a the contour Of constant U E - G} are circles. This also follows from 
the circle-preserving property of stereographies projection* 
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ContQura of (ilE^G) 1 , The contour interval a are .1 units. 
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GRADJEHT-SPACE I MAGE F&R A METALLIC SURFACE: 



For specular reflection we must have two constraints satisfied; the incident 
angle has to equal the omittance angle, and She Incident ray f the emitted 
ray and the Surface normal have to be COpl-anar, 



If i = e h then I = E and so {I + p p + q q)Ed = E 



And so, 



i) = A * V* + 



(i + p 5 p +■ q s q) = vi + p-- + <j s 



Next we must have (p v q*-l),. £p s± qi £ f-1)* and [0,0,-1) co-planar. That Is, 
the dot-product of any one with the cross-product of the other two must 
equal zero. Expressed another way, we must have the volume of the parallele- 
piped defined by the three vectors equal zero, Or, finally: 



det 



p q 




a that Is* p q - q p ■ 0. 



The same result could be arrived at in a more round -about fashion by requiring 
that i + e = g, and then expanding cos{i + e) - casf.9). We now have two linear 
equations in p and q: 



+ qq = /l+p£ + 



P S P + Q^ 



q s P - p 5 q - 
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Solving for p and qi 



p 

p = (•'i +■ p^ + 9^ -1) p s /(p, 4 +\ 2 )- p s TTc - cosB/fi - g)/(TT"g} 



c 



q - {/l + ^ + q ^ -I) q /{p ^ + q 2 ) = q J-^ m ^n9AT=~^7WT^ 



Tli is related to the ha [f -angle formula: 



tanpja) = /(l - C0Sa)/(l + cosa) 



S TEREOGRAPH I C PROJECT tOW: 

The gradient-space ffflage. For a metal I fc object 1$ a stereo-graphic projection 
o r :~<: iur round of the oL:jfcr. I ha t is, the sphere of sossible direct lor IS 
seen by the object Is napped Onto a place, with- the center of projection at 
On* pole of the sphere and the plane tangent at the other pole. The mapping 
is con forma I; that is ? angles ace preserved. Circles on the sphere are napped 
mapped into circles on the pi arte. The following illustrations from pages 248, 
252 T and 253- of Hilbert I Cohn-Vossen [3] will Illustrate: 
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F]a. 944a 




F]a. EtiL 
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A USEFUL DISCRIMIHANT; 

The incident, em i t ta n ce , and phase angle form a spherical triarcls .irul 
have to satisfy Certain constraints ™ that is, we cannot arbitrarily choose 
i h e h and n. The sun of any two has to exceed the third. TnTs is analogous 
to a similar result for the sides of planar triangles, U Ts easy to see 
that only one of (lie throe constraints tsn fall at any one tTme [41 , Suppose 
Tt is the following: 

i * c < g, then cos ( I + ft) > COs(g) 

5 free cosine is mono ton icaHy decreasing Tn the range to w. Expanding, 

one gets; 

cos(i)tos{e) - cos{g) > 5in(]Jsln{«] 

The righthand side is positive, so we can square both sides, hence: 

OE - fi> a > (T - i*Kl - E*> or 1 + 2I£G - [1= + E* + C 1 ) < 0. 

The symmetry of this expression suggests that we would have obtained the same 
result if we bad picked either of the other two constraints. In fact, it is 
easy to show that if i , e , and g can form a spherical triangle, Chert 

1 + 2IEG - (I 2 + |2 4 e aj > g 
and that this expression is less than zero otherwlse r 




Contours of the dlK'fminant I + 2IEG - (i^ + E 2 + G 2 ), The contour 
intervals are .05 unfts, 
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TH E AZIMUTH ANGLE: 



local 
normal 




A useful quantity for Some menip-ulatoi-S Is the azimuth angle A h between 
the projections of the Incident and emitted ray4 onto the object's surface, 
Applying a reauft of splierrcal trigonometry, we ffnd 



CCS 



{g3 - cos ( i ) cos f,e) + sin(Osin)[e)cog(A) 



So r 



cos (A) - 



G - IE 



/I - I y /pnr 



Mow Obviously* cos (A} 5 1 

Expanding, <G - lE) a * (1 - I 2 )(l - E 2 J 

And £* again. 



I + 2 EG - (I 2 + t 3 + G 2 ) 2 
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Contours of cos(A). The contour fntervals are .1 units. 
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EXPANDING THE DISCRIMINANT: 



We would like to eypresi the discriminant fn terms af p F q, p . and q , 

s ?. 

We will need the following; 



G = 1//| +V i + q 2 P and E - l//| + p z + q r 
L*t X = (I + p p + q q ) t then 1 ■ XEG 

Th*rt F t + 21 EG - (I 2 + £ 2 + & 2 ) = 1 + 2j(E 2 C 2 - [X 2 E 2 S 2 + E E + G 2 ) 

= -E 2 G 2 + 2XE 2 & 2 - K 2 E 2 G 2 + | + E 2 G 2 - E 2 - G 2 

- -E^l - X)Z + (1 - E±)(| - El) 

= tO/E 2 - OO/G 2 - I) - (1 - X) 2 ]E^ 

- [Cp 2 + q 2 Hp 2 + q 2 ) - ( P D + q q) 2 ) £ 2 G 2 

- Ca s P - P $ q} 2 E 2 & 2 (13 

it is i [mediately apparent that the discriminant is positive for all points 
in gradient- space, as it should be. But what Is Ittore exdt Tng [s that we 
f»vm an equation that is linear In p and q and thus helpful If we arc going 
to try to obtain p and q, given I, E, and G: 

q p - p q - ±/| + 2IEG - lt r + E^ + G 2 ) /EG 
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FINDING p AND q. GIVEN I. E, AND Gj 

We no* have CMC linear equations In p and ,, «.« from the express for I 
the other from the expand Ton of th* discriminant: 

P S P + q g q - (l/E - E)/e 

q & p - p^q - ±(4/E)/G 

where ^=[+2lEG-fl i +r 2 * r* 2 1 e ■ i * 

<IJW <| + E + & J. Solving for p and q we get: 

P ■ (l/E - Sj/S^/Cp^ 4 q ^ ) ± (VE)/G(| 
fl=fT/E - G>/Gq /(p 2 + q 2; ± ( VEj/Sp 



5 %r * " S 

(P S Z ♦ q a ) 



P - q. 









P = P p e&s(e) - q" siri(&) 
q - p 1 sln(a) + q' cos(e) 
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DETERHIhATJPH OF ORIEHTATIQH OF PLANES: 



An example will illustrate how image Intensity information can augment 

HacJcwoxth's gradTent space scheme for interpreting polyhedral scenes. 
We are given a trihedral tOrner projected into the image as follows: 





The corresponding grad I enE-space digram with position and scale as yet 
undetermined Fs on the right. »w we arc cold that the (normal ized) Image 
intensities are ,79 T r %Q r and .B6 for the regions A f B p and C respectively. 
We thus have six constraints an the position of the three points in gradient 
space. Given that (P^qJ - (0,7,0.3), and *(l P E,G) - 1, we tan develop 
the following equations? 



% = \ 



{ «A " %} ' *T Cp fl " P B ) 



(q A " q c ) - -/f (p A - P C ) 
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(1 + ,7P A + -3q A ) - /TtP^tV ^ 

(I + ,7p B+ .3q B ) = /I + V + q B ^ 7^ 

■ 

Where we used the fact that G " .30, Squaring the second set of three 
■equations, we obtain second-order polynomials, This simply reflects the 
fact that the points are constrained to He or certain conic sect ions - 
Using an Iterative mod if led Hewton-lUphsori method!, one quickly converges 
to a solution as follows; 



<P A .q A ) - {0. .70] 



(p B ,q ft ) s (-.61. -,35) 



i? c .\) - (*-6l, --35) 



The polynomials are actually simple enough that they might be solved 
d " _ c c L ' y u ;; " ig appropriate s yr bo '. r a n " p u ' a 1 1 en a 1 g c - i t h n s , 

The following questions are left a<; an exercise for the redd«rr 

1. Is there another solution? 

2. Are there solutions for which the three edges are 
concave? 

3. Are the points G G as precisely determined in 
gradi Cent- space as the soiit 1,? 
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oration of the constraints on the three g radl en t-space 

•nd G . Th[s ii One aorutfon tc the problem of 
oriefttatio.ni Of the three faces meeting at the 



points (m 



ft* 



detferahinTng the 
corner. 
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MJTUAL ILLUMINATION. — P LAMES TRUNCATED IN ONE DIRECTION 




In order to get a feel for the mutual illumination problem it helps, to study A 
simple cast first. Consider two planes joined at right angles,, infinite in the 
direction of the line of their intersection, and both of length L in the direc- 
tion away from their intersection. Let the incident light come frotn a distant 
source and in a direction ir/4 with respect to the planes. This last condition 
and the equal lenth of the sides provide the symmetry necessary to ensure that 
the intensity distribution on the two planes is equal. Next we will assume 
that the surfaces are lambertian* with reflectivity r, 
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Now let us calculate the total light flu* deceived by a surface element d 
distance x away frcKn the earner. First consider the COfl 1 1 i bti 1 1 on due to a 
Syrfatt; clement on the other plane a distance V from the corner and -separated 
by a distance z along the direction of the line of intersection. Let the 
luminous effiittancc vary as Lfy). Then this contribution will be, for 
lambertlan surface* h 

(r/Tr)[co*(f)cos(e}]/l 2 L(v)dydz (Flux/unit area) 

Htfc 1 is the distance between the two point*, e is the angle of emittance 
at the emitting, surface element and \ is the angle of incidence at the re- 
ceiving surface element. 

COs(«) * fc/1 , cos(l) = y/1 N and \ z =■ x z + y 2 + z 2 

So the contribution due to the patch (dy by dz) Is then: 

L(y)dy(r/ir>6cy)/l*di 

integrating with respect to z + one obtains^ 

My)dy(r/Tr)xy I l/(* 2 + y* + z^} 2 dz 
Mow, J 1/^ + S 2 ) 2 ds = (l/ a »>[ir/2) 

So we jet; L(y)dy hr xy/(x 2 + y 2 ) V 2 
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NTEGML EQUATION: 



Finally Integrating wtth reiOect to y and adding In the dfr*ct contr Ebilt ion : 



tfn) = E//T + ^ r / L ^ Lfy}dy 

(* 2 + y J ]V2 



So here we have an Implicit equation for L(x) called an Integral equation. 
Before we try to so We It, notice that the parameters £ and L can be eliminated, 
For example, if L(jcJ Is a solution for incoming light flux E. then aL(x) will 
be a solution if the light-flux is changed to aE. That Is, everything just 
gets brighter in proportion, if we increase the incident flux. Without a 
loSi of generality, we can set E^*2 = I. 
Weat, let x' - x/L an y 1 = y/L, t'iftn w* find 



< lu ■ 



L(x') = I + 1/2r /' *-* L y J )<Jy' 

d f , 2 .V 3/z 



So, we can, without loss, of generality* also let L = 1. So we will try to 
solve: 



1 xy 



L(k) - I + hr r ( k ; . 7 3)3 77 MvMy 



Thib is a Fredrmlm integral equattor iy" : -,c seco-.c k i -id [lo], 5-ich f.:\.i-:n. i ms. 
usually occur as solutions to ordinary differential equal tons, with given 
end conditions. The kernel is, 
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KU,yJ - ** 

{x 2 + yi)3/l 



Th* kernel is symmetric, non-separable, and worst of all, not bounded. There 

are a number of technics for solving such equations with symmetric kernels, 
but most work only f or bounded kernels, or. if one can calculate the iterated 
kernel . 
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iterative SOLUTION: 



One method i& iteration [lOj. Suppose we start with L (*} =0. Substituting 

s 

this In the r I g h J hand side of the equation we arrive at the next approxfmat Ton : 
L|(^} = 1. Using this we integrate again and find: 



UU) = 1 + H r(1 - 



/l + X 2 



The next step leads to: 

L (x) = 1 + h r{l + ^r)(l ^— ) - ( h r) 2 xf 1 ! ~ — ^ J 

This last terra turns out to he some messy dtffcrenc* of elliptic integrals 

and SO we abandon further iteration. A few things of note emerge, however. 
First of all, wc have a useful first approximation in L^x) or the first few 
terml in Lj(x) L This approxlmat Ion is particularly good far small r, since 
the remaining omitted terms are in r 2 and higher orders. 

Secondly* the leading tern will clearly become on Further iteration: 
1 + ^ r + [ ij r j? + [ !j r )i + ,._ 1/(1 _ ^ p ) 

And so, L{0) = 1/(1 - -^ rj , not too surprisingly, Hext one can say some- 
thiing ahout the convergence of this iterative process -- It will converge for 
r less Lhan 2, and diverge fof r greater than or equal to 2- Obviously, we 
care only about values for r between iero and one, so we expect a solution 
wi 1 I always exfat. 



dy 
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IWTEGRATION BV PARTS: 

Furtber useful remits can be obtained by integrate h v partS . 

M«J - 1 + I"S r IL(O) - LCI) — * + r' L'fyJ 



^' + ** ° V^T^z 



tfy] 



Clearly L { J - ] + , h r Mo) , nd ^ 0nce ^^ ^ ^ ^ ^ _ 

1/0 ~ i%r), Thf, result depends on the fact that the integral j s rero for 
a - 0> which tan h* 5hQ wn by applying L'Uwpital's rule to the resuTUng 

Ind.terinJnate 1W 6y a Codious melh od of Htt)e f|lt|!nsit ^ ^ ^ 
arrive at another approximation: 

L(k) = | + V _ m _ (I - ^r), 

1 - %r L J 

This appro, lotion is particularly good near the orFgm and can be "tuned" 
by nuUFplyh* the factor containing x by a number sm ,U*r than one. The 
form of thU r.^lt sk™ 5 that Lfc) will h«« a Ctltp , t tne OTlff)n for r 
greater than or equal to one. 
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MU.HERKAL SOLUTION: 

It r& becoming increasingly ohvlous that an inalytleal solution is not 
around the corner, SO it is time to turn to numerical met hods- * The. re. art-. 
ayain various possible avenues, thE most obvious being iteration — since we 
already know some good First appnoxl mat Ions we can speed up the convergence. 
The only difficulty i-j the singularity in the kernel for * - y - 0, Dividing 
the range of Integration evenly produces quite poor results particularly near 
the origin for large values of r. Dividing the range more finely near th* 
origin and ignoring the first few values near there is the obvious solution. 
Choosing as end-points of the intervals the points (i/n) works quite welt. 
Here n is the total number Of Segments En the interval fron to k The 
mid-points of the intervals are used when evaluating the kernel. 
The resulting solutions for n = Z5& were presented -graphically earlier. 



